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Maintenance  Questionnaire  and  List  of  Respondents* 


B.  P.  Lientz,  E.  B.  Swanson,  and  G.  Tompkins 
Graduate  School  of  Management 
University  of  California,  Los  Angeles 


Abstract 


This  paper  contains  the  maintenance  questionnaire 
and  list  of  respondent  organizations  for  a 
maintenance  survey  conducted  in  1976. 


♦This  work  was  partially  supported  by  the 
Information  Systems  Program,  .Office  of  Naval 
Research  under  contract  NOOO  14-75-C-0266, 
project  no.  NR  049-345. 


1.  Introduction  and  Summary 


There  has  been  an  Increasing  concern  with  the  operational  phase 
of  the  life  cycle  of  software  systems.  This  phase  is  concerned  with  the 
maintenance  and  enhancement  of  computer  based  Information  systems. 

Maintenance  Is  the  process  of  having  the  system  meet  the  requirements 
specified  In  development.  Enhancements  are  changes  made  to  the  system 
to  meet  new  requirements. 

To  analyze  these  processes  a questionnaire  was  constructed,  field 

("Tin*  — >. 

tested,  and  finalized.^  A copy  appears  In  section  2.  The^questlonnalre,  u>kitk 

Is  divided  Into  two  parts:  one  dealing  with  the  systems  data  processing 

and  programming  department  and  the  other  with  a specific  application 

system  being  maintained  and  enhanced.  The  questionnaire  Includes  Items 

of  both  technical  and  managerial  Interest. 

The  questionnaire  was  distributed  to  120  organizations.  Responses 

" j. — t 

were  collected  for  69  applications^  “The  organizations  respondimfSre 

listed  In  section  3.  * 

V 

The  analysis  of  results  and  existing  literature  will  appear  In 
separate  papers. 
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2.  Questionnaire  on  Application  Software  Maintenance 


The  actual  questionnaire  begins  on  the  next  page. 


Space  tor  "tesarks"  is  also  frequently  included,  where  you 
feel  that  your  answer  can  be  usefully  clarified,  please  enter 
your  resarks  acccrainyly.  It  is  iaperative  that  every  question 
be  answered. 


Your  assistance  ana  cooperation  is  greatly  appreciated 
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To  what  industry  does  tue  organization  served 
aeparteent  belong?  (Check  the  one  which  aost 
applies. ) 

fl anutactur ing  industries: 

a.  Data  Processing  Equipment 

b.  Instruaents/Electrical 

c.  Che«ical/Alliea  Products 

d.  Printing/Putlishing 

e.  Food/Tonacco 

t.  Priaary/Pabricated  Metal 

g.  Trans  per taticn  Equip Bent 

h.  Petroieue/Ccal/Butber  . 

i.  Paper/taper  Products 

j.  Textiles/Apparel 

k . Other  (Please  indicate)  _ 

N on-ian utactur ing  industries: 

a.  Insurance 

b.  Banking/Credit  Agency 

c.  EDP  Services 

d.  Education 

e.  Government 

£.  Public  Utility 

g.  Investaent 

h.  Mining/Ccnstr ucticn 

i.  Transportation 

j.  Consultants 

k.  Other  (Please  indicate)  


by  your 
closely 


i 


j 
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; 

t 


that  is  the  annual  budget  of  your  company  for  data 
processing  equipment  rental,  maintenance,  and  amortization 
expense?  (Check  the  category  which  applies.) 


a.  $2,000,000  or  more  I _| 

b.  Less  than  12,000,000, 

but  $1,5CC,0G0  or  more  I I 

c.  Less  than  $1,500,000, 

but  $1,000,000  cr  more  | | 

d.  Less  than  $1,300,000, 

but  $500, COO  cr  more  I ...  I 

e.  Less  than  $500,000, 

tut  $250,  000  cr  more  I _| 

f.  Less  tnan  $250,000, 

but  $125, 000  or  more  I | 

g.  Less  than  $125,000  I I 


What  is  the  total  number  cf  personnel  in  your  Systems  0 
Programming  department? 

Ct  the  total  number  cf  personnel,  how  many  »re  employed  as 
systems  analysts  and/cr  programmers? 


Of  the  total  number  c£  systems  analysts  and  prog  rammers,  how 
■any  worm  in  -jach  of  the  following  categories: 


I Program-  | Systems  | Both  i 
t sing  but  | analysis,  | program-  | 
I not  j but  not  | Bing  and  | 
I systems  | program-  | systeas  | 


I analysis  | aing 


| analysis  | 


New  application 
system  development, 
but  net  application 
systea  maintenance 


Application  system 
maintenance  but  not 
new  application 
system  development 


Both  new  application 
system  development 
and  application 
system  maintenance 


Neither  new  applica- 
tion systea  develop- 
ment nor  application 
systea  maintenance 
(e*  g.  systeas 
programming) 


hsaarns 


" 

I • 

o.  Witniu  your  department,  ate  formal  lines  of  authority  and 
responsibility  drawn  sc  as  tc  separate  the  management  cf 
programming  from  the  aanageaent  of  other  activities? 


Beaarks: 


Tes:  | | 


7.  Within  your  Systems  6 Prcgraaming  department, 
lines  or  authority  and  responsibility  drawn 
separate  the  aanageaent  of  application  system 
troa  the  aanageaent  of  other  activities? 

Yes:  | | 

Heaarks: 


No:  | | 


are  fornal 
so  as  to 
maintenance 


\ 


In  terms  of  the  total  {erscn-houcs  Mocked  annually  by  youc 
programing  and  systems  analysis  personnel,  what  percentage 
is  spent  in  each  or  the  following  activities: 


Application  system  maintenance 


New  application  system  uevelcpment 


Other 


TOTAL 


Hemarks 


In  your  judgment,  how  important  are  the  problems  which  arise 


n application  system  maintenance  when  compared  to  those  of 


new  application  system  development?  (Check  one  of  the 


following.) 


a.  Maintenance  problems  by  far  the  more  important 


b.  Maintenance  problems  somewhat  mere  important 


Maintenance  and  new  system  development  problems 
of  egual  importance 


New  system  development  problems  somewhat  more 
important 


New  system  development  problems  by  far  the  more 
important 


10.  Suppose  that  the  progra uni ng  and  systems  analysis  capacity 
of  youx  department  ccula  be  increased  by  10*  (in  terms  cf 
the  number  or  equivalent  persons  available) . Assuming  that 
needed  skills  are  reflected  in  the  increase,  how  would  ycu 
allocate  this  additional  capacity? 


Application  system  maintenance: 
New  application  system  development: 

Other  Activities: 


TOTAL:  10K 


Suppose  instead  that  the  capacity  could  be  increased  by  25*. 
How  would  ycu  allocate  this  larger  percentage  increase? 


Application  system  maintenance: 
New  application  system  development: 


Other  Activities: 


TOTAL:  25* 


11.  Suppose  that  the  prcgraaairg  and  systeas  analysis  capacity 
o£  your  depactaent  was  to  te  decreased  by  10%  (in  terns  of 
the  nuaber  of  equivalent  persons  available).  Assuaing  tnat 


you  couxd  ueteraine  where  the  cut-back  was  to  be  aade,  how 


would  you  allocate  the  reduction? 


aaintenance 


New  application  systen  develcpaent 


Other  Activities 


TOTAL 


Suppose  instead  tnat  the  capacity  was  to  be  decreased  by 


How  would  you  allocate  this  larger  percentage 


reduction? 


Application 


New  application  systea  develcpaent 


Other  Activities 


. 

9 * 

* 

12.  helative  to  the  tables  at  hand,  hoe  would  you  evaluate  your 
current  level  ot  i-royraaainy  and  systees  analysis  staffing, 
on  the  whole?  (Check  cne  cf  the  following.) 


a.  Substantially  understaffed:  | 

b.  Sosewhat  understaffed:  I 

c.  Neither  understaffed  nor  overstaffed:  i 

d.  Sosewhat  overstaffed:  | 

e.  Substantially  overstaffed:  | 


Resarks: 


Please  identify  an  application  system  maintained  by  your 
department  ehicii:  (i)  has  been  operational  one  year  or 
longer;  (ii)  represents  a significant  investment  of  time  and 
effort  by  ycur  department;  ana  (iii)  is  considered  by 
management  to  be  of  fundamental  importance  to  the 

organization. 

Name  of  application  system:  

basic  function  (Describe  briefly): 


«adt  organizational  units  are  the-  primary  users  ot  the 
application  system?  (A  "primary  user"  is  uefined  here  as  a 
unit  Khicn  receives  outputs  from,  or  provides  inputs  to  the 
application  system.) 


to aa t is  the  total  number  of  personnel  in  the  primary  user 


units  , taken  together? 


I £&egk  i h3  appligab±e  sial^mejjt:  ~1 

I I 

| i'he  above  answer  is:  I 

j Seasonably  accurate,  based  on  good  data  | 

I A rough  estimate,  based  on  minimal  data | 

| An  estimate,  not  based  on  any  data___  | 

i ZZ i 


Of  the  total  number  of  personnel  in  the  primary  user  units, 
what  percentage  is  actively  engaged  in  receiving  the  outputs 
or  providing  the  inputs? 


% 


I £&££&  ifcS  diiiicable  statement: 

I 

| The  above  answer  is: 

j Reasonably  accurate,  based  on  good  data 

| A rough  estimate,  based  on  minimal  data 

| An  estimate,  net  based  on  any  data 


On  what  date  (month  and  year)  did  the  application  system 
become  operational? 


Year 

Month 


Remarks 


uhat  is  th*»  total  nuauer  of  prograss  in  the  application 


syste*  aaintamed?  (ihe  tars  "progras"  is  associated  here 


with  a olcck  ot  source  language  stateaents  coapiled  cr 
asseaoled  as  a unit.) 


| The  aocve  answer  is: 

| Reasonably  accurate,  based  on  good  data. 
I A rough  ostisate,  based  on  niniaal  data. 
| An  estimate,  net  based  or.  any  data. 


tanat  is  the  total  nuaber  cf  source  language  stateaents 


(excluding  ecssents)  incluceu  in  the  application  systea 


saintainod? 


I atii i£ab±e  st^tesent 


I The  above  answer  is: 

| Reasonably  accurate,  based  on  good  data. 
| A ..cugn  esticate,  based  on  sinisal  data, 
j An  estimate,  not  based  on  any  data 


STS. 


I 

i * 


9.  Of  the  total  nuaber  ci  source  language  statements 
■aintained,  what  percentage  originated  in  each  of  the 
following  tiae  periods? 


1975  - 76 

% 

1973  - 74 

% 

1971  - 72 

* 

1909  - 70 

* 

1967  - 68 

% 

1966  or  earlier 

% 

TOTAL 

loot 

I iliSS*  the  ailiicdbie  statement: 

I 

| The  above  answer  is: 

| neasonably  accurate,  based  on  good  data _ 

| A rough  estimate,  based  on  aininal  data 

j An  estittate,  net  based  on  any  data” 


10.  Of  the  total  nuaber  ct  source  language  statements 
aaintainei,  what  percentage  is  associated  with  prograss 
which  carry  cut  online  processing? 

X 


I Qh*£*  Statement: 

I 

| Ihe  above  answer  is: 

| Reasonacly  accurate,  based  on  good  data 

I A rough  estimate,  based  on  sinisal  data 

| An  estisate,  not  based  on  any  data__3 
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13.  Please  identify  the  following  characteristics  of  each 


r 


ccaputer  systes  - hardware  and  software  - employed  in  the 
operational  processing  of  the  application  systes: 


Manufacturer  (s) 
Model  (s)  


Hon th/year  when  nodal  (s)  became  operational  

Operating  Systes  Software  


14.  Is  distributed  processing  employed  in  the  processing  of  the 
application  system?  (The  term  '’distributed  processing” 
implies  segmenting  data  bases  and  distributing  application 
software  processing  aacng  computers  at  different  locations.) 


Yes:  | | 


No: 


I I 


I 1 

I 

I 

ft 


15.  Is  a data  base  management  system  employed  in  the  processing 
of  the  application  system? 


Yes:  | | No:  | | 


It  so,  which? 


1o.  How  aany  individual  data  files  aake  up  tha  data  base 
associated  with  toe  application  systea?  (The  tera  "data 

base"  is  defined  here  siaply  as  the  set  of  easier  files 


i- 

♦ 


k. 


associated  with  a systea.) 


fieaaiks: 


17.  boat  is  the  average  size  ci  the  data  base,  aeasured  in  total 
nuaber  of  bytes  (or  the  equivalent) ? 


I £JlS£*  thi  siiteaext: 


I 


( The  aoove  answer  is: 

| heasonably  accurate,  based  on  good  aata 

| A rcugh  estiiate,  based  on  ainiaal  data 

| An  estimate,  not  based  on  any  data... 


20 


18.  0 1 the  total  nuaber  ot  bytes  contained  in  the  data  base, 
waat  percentage  is  the  result  .'£  an  update  aade  during  each 
oi  the  iollcwiny  tine  periods,  cn  the  average? 


The 

acst 

recent  day: 

% 

Net 

the 

aost 

recent 

day. 

but 

the 

aost 

recent 

week: 

% 

Net 

the 

aost 

recent 

week. 

but 

the 

aost 

recent 

aonth: 

% 

Net 

the 

aost 

recent 

month. 

but 

the 

aost 

recent 

guarter : 

% 

Not 

the 

aost 

recent 

guarter. 

out 

the 

aost 

recent 

year: 

S 

Not 

the 

nos  t 

recant 

year: 

% 

TOTAL  100* 


| Che^k  £he  aj^iic^ble  stateaent:  I 

I ” I 

| The  accve  answer  is:  1 

| Reasonably  accurate,  based  on  good  data I 

| A rouyh  estiaate,  based  on  ainiaal  data | 

| An  estiaate,  not  based  on  any  data._.  I 
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20.  Of  the  pre-uefined  user  reports,  how  many  are  issued  in  each 


of  the  follcwing  forms 


Visual  display  cr  printed,  on  deaand 


Visual  display  cr  printed,  daily 

Visual  display  cr  printed,  weekly 

Visual  display  cr  printed,  Monthly 

Visual  display  cr  printed,  quarterly 


Visual  display  cr  printed,  se«i-annually 


visual  display  cr  printed,  annually 


Others  (Please  indicate) 


| The  above  answer  is: 

| treasonably  accurate,  based  on  good  data 
I A rough  estimate,  based  on  minimal  data. 
| An  estimate,  net  based  on  any  data. 


Which  of 

the  following  tools  and  techniques  were  employed  in 

the  development  of 

the  application  system?  (Check 

those 

which  apply.) 

a. 

Decision  tables 

1 — 1 

b. 

Data  base  dictionary 

1 — — 1 

c. 

lest  data  generators 

1 • 

d. 

Structured  programming 

• ...  I 

e. 

1SL0S  (Information  System  Design 
and  Optimization  System) 

| , ni  | 

«- 

a.  • 

Chiet  Programmer  Team 

1 1 

g* 

On-line  Programming 

i — -1 

h. 

Automated  flowchart ing 

1 ...  1 

i. 

Hlpo  (Hierarchy  plus  Process- 
Output)  Design  Aid  Technique 

j* 

Structured  Walk-thru 

1 ...1 

Others  (Please  indicate) ; 

k. 

1. 

m. 

What  is 

the  total 

number  cf  person-hours  now  e 

xpended 

annually 

on  maintenance  ol  the  application  system? 

I £iiS£K  ‘Jlliicable  statement: 


| The  above  answer  is; 

| Reasonably  accurate,  based  on  good  data 
| A rough  estivate,  based  on  minimal  data 
| An  estimate,  net  based  on  any  data 


Of  the  total  number  of  p j rson-hours  now  expended  annually  cn 
maintenance  of  the  application  system,  what  percentage  is 
expended  in  each  of  the  renewing  problem  areas: 


Routine 


Accommodation  cf  change 
data  inputs  ana  tiles 


Accommodation  of  changes  to 
hardware  and  system  software 


t.  Improvement  of  program  documentation 
g.  Recodinc  for  efficiency  in  computation 


(Please  indicate) 


Otlie  r 


IOTAL 


t Chech  £ he  applicable  statement 


| The  above  answer  is: 

1 Reasonably  accurate,  based  on  good  data. 
| A rough  estivate,  based  on  minimal  data 
| An  estimate,  not  based  on  any  data 


Of  the  total  number  of  person-hours  now  expended  annually  cn 
maintenance  of  the  application  system,  what  percentage  is 
spent  cn  maintenance  of  programs  which  perform  online- 
processing? 


I £ijeck  ihe  apj^icatls  ii ai£»§I»t: 

I 

| Tha  abcve  answer  is: 

| Reasonably  accurate,  based  on  good  data, 
| A rough  estimate,  based  on  minimal  data, 
| An  estimate,  net  based  on  any  data. 


Of  the  total  numoer  of  persen-heurs  now  expended  annually  cn 
maintenance  et  the  application  system,  what  percentage  is 
spent  in  communication  with  the  user? 


| Check  _£he  a££l±cable  statement: 

I 

| The  above  answer  is: 

| Reasonably  accurate,  based  on  good  data 

| A rough  estiratc,  baseu  on  minimal  data 

| An  estimate,  not  based  on  any  data 


What  is  the  total  number  of  individuals  currently  assigned 
(in  whole  or  in  part)  to  maintenance  or  the  application 
system? 


29.  Or  the  total  nuaber  ot  individuals  currently  assigned  to  the 
maintenance  of  the  application  systea,  hou  aany  have  each  of 


the  following  levels  of  on-the-jcb  programming  experience? 

a.  five  years  cr  acre:  

i>.  Less  than  five  years,  but  three  or  aore:  

c.  Less  than  three  years,  but  one  or  aore:  

1.  Less  tnan  one  year,  but  soae:  

e.  None  of  the  aocve: 


heaarks : 


JO.  Are  all  changes  Bade  to  the  application  system  programs 
logged  and  documented  according  to  an  established 
organizational  procedure? 

Yes:  | | No:  | | 

Now  aany  changes  were  recorded  during  the  past  twelve 
months? 


Are  all  user  requests  for  changes  to  the  application  sys tea 
logged  and  documented  according  to  an  established 
organizational  proceaure? 

*es:  | | Mo:  | | 

How  nany  reguests  were  recorded  during  the  past  twelve 
aonths? 

of  the  reguests  reccroed  duiing  the  past  twelve  aonths,  hew 
inany  have  new  been  satisfied  by  changes  made  to  the 
application  system? 


Hemarks: 


Are  all  troubles  encountered  in  the  operational  processing 
ot  the  application  system  programs  logged  and  documented 
according  to  an  estaolished  organizational  procedure? 

*ss:  | | No:  | | 

How  many  trouble  reports  were  recorded  during  the  past 
twelve  months? 


JJ.  Is  a foetal  audit  of  the  application  system  made 
periodically,  according  to  an  established  organizational 
procedure? 


Yes:  | | No:  | | 


If  so,  how  rreguantly,  ou  tbs  average? 


J4.  Is  the  documentation  associated  with  the  application  system 
programs  maintained  by  a program  or  system  librarian? 


1 


Yes:  | | No:  | | 


Remarks: 


30 


Is  an  accounting  procedure  employed  to  accumulate  the 
personnel  coats  in  your  department  associated  with  operating 
and  maintaining  the  application  system? 

Yes:  | | No:  I I 

11  so,  what  was  the  total  personnel  cost  (in  dollars) 
accumulated  during  the  past  twelve  months? 

henarks: 


Is  an  acccuatiug  procedure  employed  to  accumulate  the 
eguipment  costs  in  your  department  associated  with  operating 
and  maintaining  the  application  system? 

Yes:  | No:  | | 

If  so,  what  was  the  total  eguipment  cost  (in  dollars) 
accumulated  during  the  past  twelve  months? 


Remarks: 


i 7.  Are  the  personnel  ana  eyuip*ent  costs  associated  with 
operating  and  Baintainiuy  the  application  systea  charged 


back  (in  whole  or  in  part)  to  the  user? 

Yes:  I I Mo:  | I 

II  so,  cu  wbat  basis  is  the  charge  Bade?  (Check  one.) 

a.  £guipaent  utilization  exclusively:  I J 

b.  Personnel  utilization  exclusively:  I , I 

c.  Both  eyuxpiect  and  personnel  utilization:  I m . .1 

d.  Neither  «yuipm»nt  nor  personnel  utilization:  |___| 

If  costs  are  charged  bacx  to  the  user,  what  was  the  arcuct 
charged  during  the  past  twelve  tenths? 


Ho Barks: 


I ' 


3d.  Iii  your  judgment,  to  what  extent  have  the  following  been  (cr 
are)  rroulems  in  maintaining  the  application  system  you  have 
described? 


None  At  All 


Soaewhat  Minor 


Minor  Problea 


Soaewhat  Major 


Kajor  Problem 


Turnover  of  Maintenance 
personnel 

quality  or  application  system 
documentation 

Changes  made  to  system  hardware 
and  software 

User  demand  ror  enhancements  and 
exteusicrs  to  application  system 

Skills  or  maintenance  programming 
personnel 

quality  or  original  programming 
of  application  system 

Number  or  maintenance  pregramming 
personnel  available 

Competing  demands  for  maintenance 
programming  personnel  tixe 

Lack  of  user  interest  in 
application  system 

Application  system  run  failures 


k.  Lack  or  user  understanding  cr 
application  system 


}■ 


3 J 


None 


At  All 


Scaewhat  Minor 
Minor  Problea 

— i 


Soaewhdt  Major 
Magee  Problee 
~l  5 


Storage  requirements  c £ 
application  system  pregrams 

Processing  time  requirements  of 
application  system  programs 

Motivation  of  maintenance 
programming  personnel 

forecasting  of  eaircenance  pro- 
gramming personnel  requirements 

Maintenance  programming 
productivity 

System  hardware  & software 
reliaoi lity 

lata  integrity  in  application 
system 

Unrealistic  user  expectations 


Adherence  to  programming 
standards  in  maintenance 

Management  support  cf 
application  system 

Adequacy  cf  application 
system  uesign  specifications 

Budgetary  pressures 


Meeting  scheduled  commitments 
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3.  Responding  Organizations 


Below  Is  a list  of  the  organizations  that  responded  to  the 
questionnaire. 

1.  American  Automobile  Association 

2.  Architects  Limited 

3.  Atlantic  Richfield  Company 

4.  Avco  Financial  Services 

5.  Bankamerlcard  Systems 

6.  Bank  of  America 

7.  Baskins  and  Robbins 

8.  Bell  Industries 

9.  B.F.  Goodrich  Company 

10.  Brazilian  Systems , Inc. 

11.  Burroughs— Management  Services 

12.  Burroughs  Corporation— Western  Application  Development  Center 

13.  Byron  Jackson  Pump  Division 

14.  City  of  Los  Angeles 

15.  Coast  Community  College  District 

16.  Firestone  Tire  and  Rubber  Company 

17.  Ford  Motor  Company 

18.  Franklin  Computer  Services 

19.  General  Motors  Corporation 

20.  Getty  Oil  Corporation 

21.  Gibraltar  Savings  and  Loan  Association 

22.  Glendale  Federal  Savings  and  Loan  Association 


23.  Great  Western  Savings 

24.  Hawaiian  Telephone 

25.  H.C.  and  D. , Limited 

26.  Honeywell  Information  Systems 

27.  Hughes  Aircraft 

28.  Hughes  Airwest 

29.  Hughes  Markets,  Incorporated 

30.  Island  Insurance 

31.  Knape  and  Vogt  Manufacturing  Company 

32.  Knudsen  Corporation 

33.  Lever  Brothers 

34.  Litton  Industries 

35.  Lloyd's  Bank  of  California 

36.  Los  Angeles  City  Schools 

37.  Los  Angeles  County 

38.  Los  Angeles  Department  of  Water  and  Power 

39.  Los  Angeles  Times 

40.  Market  Basket 

41.  May  Company 

42.  NCR 

43.  Northrop  Corporation 

44.  Occidental  Life  Insurance 

45.  Pennsylvania  Life  Insurance 

46.  Pertec 

47.  Petrobas 

48.  Price  Waterhouse  and  Company 


49. 

50. 

51. 

52. 

53. 

54. 

55. 

56. 

57. 

58. 

59. 

60. 
61. 
62. 

63. 

64. 

65. 

66. 

67. 

68. 


Revel 1 Incorporated 

San  Fernando  Board  of  Realtors 

Security  Pacific  National  Bank 

Sistema  and  Computacao 

Southern  California  Edison 

Southern  California  Gas  Company 

State  of  California  Department  of  Motor  Vehicles 

Sunkist  Growers 

Thrifty  Stores 

Tobias  Kotzin 

Transamerica 

UCLA  Administrative  Center 
UCLA  Medical  School 
Union  Oil  Company 
University  of  California 
United  California  Bank 
Van  de  Kamps 
Veterans  Administration 
Walt  Disney  Productions 
Western  Airlines 


PNPun 
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